System for characterizing manual welding operations on pipe and other curved structures

ABSTRACT

A system for characterizing manual welding exercises and providing valuable training to welders that includes components for generating, capturing, and processing data. The data generating component further includes a fixture, workpiece, at least one calibration device having at least two point markers integral therewith, and a welding tool. The data capturing component further includes an imaging system for capturing images of the point markers and the data processing component is operative to receive information from the data capturing component and perform various position and orientation calculations.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is being filed as a non-provisional patentapplication claiming priority under 35 U.S.C. §119(e) from, and anyother benefit of, U.S. Provisional Patent Application No. 62/055,724filed on Sep. 26, 2014, the entire disclosure of which is hereinincorporated by reference.

The following commonly-assigned, co-pending U.S. patent application isalso incorporated by reference herein in its entirety: U.S.Non-Provisional patent application Ser. No. 13/543,240, filed on Jul. 6,2012 and entitled “System for Characterizing Manual Welding Operations.”

FIELD

The general inventive concepts relate, among other things, to methods,apparatus, systems, programs, and techniques for remotely controllingoperation of a welding device.

BACKGROUND

The described invention relates in general to a system forcharacterizing manual welding operations, and more specifically to asystem for providing useful information to a welding trainee bycapturing, processing, and presenting in a viewable format, datagenerated by the welding trainee in manually executing an actual weld inreal time.

The manufacturing industry's desire for efficient and economical weldertraining has been a well documented topic over the past decade as therealization of a severe shortage of skilled welders is becomingalarmingly evident in today's factories, shipyards, and constructionsites. A rapidly retiring workforce, combined with the slow pace oftraditional instructor-based welder training has been the impetus forthe development of more effective training technologies Innovationswhich allow for the accelerated training of the manual dexterity skillsspecific to welding, along with the expeditious indoctrination of arcwelding fundamentals are becoming a necessity. The characterization andtraining system disclosed herein addresses this vital need for improvedwelder training and enables the monitoring of manual welding processesto ensure the processes are within permissible limits necessary to meetindustry-wide quality requirements. To date, the majority of weldingprocesses are performed manually, yet the field is lacking practicalcommercially available tools to track the performance of these manualprocesses. Thus, there is an ongoing need for an effective system fortraining welders to properly execute various types of welds undervarious conditions.

SUMMARY

The following provides a summary of certain exemplary embodiments of thepresent invention. This summary is not an extensive overview and is notintended to identify key or critical aspects or elements of the presentinvention or to delineate its scope.

In accordance with one aspect of the present invention, a system forcharacterizing manual and/or semiautomatic welding operations andexercises is provided. This system includes a data generating component;a data capturing component; and a data processing component. The datagenerating component further includes a fixture, wherein the geometriccharacteristics of the fixture are predetermined; a workpiece adapted tobe mounted on the fixture, wherein the workpiece includes at least onejoint to be welded, and wherein the vector extending along the joint tobe welded defines an operation path, wherein the operation path islinear, curvilinear, circular, or a combination thereof; at least onecalibration device, wherein each calibration device further includes atleast two point markers integral therewith, and wherein the geometricrelationship between the point markers and the operation path ispredetermined; and a welding tool, wherein the welding tool is operativeto form a weld at the joint to be welded, wherein the welding tooldefines a tool point and a tool vector, and wherein the welding toolfurther includes a target attached to the welding tool, wherein thetarget further includes a plurality of point markers mounted thereon ina predetermined pattern, and wherein the predetermined pattern of pointmarkers is operative to define a rigid body. The data capturingcomponent further includes an imaging system for capturing images of thepoint markers. The data processing component is operative to receiveinformation from the data capturing component and then calculate theposition and orientation of the operation path relative to thethree-dimensional space viewable by the imaging system; the position ofthe tool point and orientation of the tool vector relative to the rigidbody; and the position of the tool point and orientation of the toolvector relative to the operation path.

In accordance with another aspect of the present invention, a system forcharacterizing manual and/or semiautomatic welding operations andexercises is also provided. This system includes a data generatingcomponent; a data capturing component; and a data processing component.The data generating component further includes a fixture, wherein thegeometric characteristics of the fixture are predetermined; a workpieceadapted to be mounted on the fixture, wherein the workpiece includes atleast one joint to be welded, and wherein the vector extending along thejoint to be welded defines an operation path, wherein the operation pathis linear, curvilinear, circular, or a combination thereof; at least onecalibration device, wherein each calibration device further includes atleast two point markers integral therewith, and wherein the geometricrelationship between the point markers and the operation path ispredetermined; and a welding tool, wherein the welding tool is operativeto form a weld at the joint to be welded, wherein the welding tooldefines a tool point and a tool vector, and wherein the welding toolfurther includes a target attached to the welding tool, wherein thetarget further includes a plurality of point markers mounted thereon ina predetermined pattern, and wherein the predetermined pattern of pointmarkers is operative to define a rigid body. The data capturingcomponent further includes an imaging system for capturing images of thepoint markers and the imaging system further includes a plurality ofdigital cameras. At least one band-pass filter is incorporated into theoptical sequence for each of the plurality of digital cameras forpermitting light from only the wavelengths which are reflected oremitted from the point markers for improving image signal-to-noiseratio. The data processing component is operative to receive informationfrom the data capturing component and then calculate the position andorientation of the operation path relative to the three-dimensionalspace viewable by the imaging system; the position of the tool point andorientation of the tool vector relative to the rigid body; and theposition of the tool point and orientation of the tool vector relativeto the operation path.

Additional features and aspects of the present invention will becomeapparent to those of ordinary skill in the art upon reading andunderstanding the following detailed description of the exemplaryembodiments. As will be appreciated by the skilled artisan, furtherembodiments of the invention are possible without departing from thescope and spirit of the invention. Accordingly, the drawings andassociated descriptions are to be regarded as illustrative and notrestrictive in nature.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and form a partof the specification, schematically illustrate one or more exemplaryembodiments of the invention and, together with the general descriptiongiven above and detailed description given below, serve to explain theprinciples of the invention, and wherein:

FIG. 1 is a flow chart illustrating the flow of information through thedata processing and visualization component of an exemplary embodimentof the present invention.

FIG. 2 provides an isometric view of a portable or semi-portable systemfor characterizing manual welding operations, in accordance with anexemplary embodiment of the present invention.

FIG. 3 provides an isometric view of the flat assembly of the system ofFIG. 2.

FIG. 4 provides an isometric view of the horizontal assembly of thesystem of FIG. 2.

FIG. 5 provides an isometric view of the vertical assembly of the systemof FIG. 2.

FIG. 6 illustrates the placement of two point markers on the flatassembly of FIG. 2.

FIG. 7 illustrates an exemplary workpiece operation path.

FIG. 8 illustrates the placement of two active or passive point markerson an exemplary workpiece for determining a workpiece operation path.

FIG. 9 is a flowchart detailing the process steps involved in anexemplary embodiment of a first calibration component of the presentinvention.

FIG. 10 illustrates the welding tool of an exemplary embodiment of thisinvention showing the placement of the point markers used to define therigid body.

FIG. 11 illustrates the welding tool of an exemplary embodiment of thisinvention showing the placement of the point markers used to define thetool vector and the rigid body.

FIG. 11A illustrates the welding tool of FIG. 10, along with anexemplary tool calibration device for interfacing thererwith.

FIG. 12 is a flowchart detailing the process steps involved in anexemplary embodiment of a second calibration component of the presentinvention.

FIG. 13 provides an isometric view of a portable or semi-portable systemfor characterizing manual welding operations, in accordance with anexemplary embodiment of the present invention.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention are now described withreference to the Figures. Reference numerals are used throughout thedetailed description to refer to the various elements and structures. Inother instances, well-known structures and devices are shown in blockdiagram form for purposes of simplifying the description. Although thefollowing detailed description contains many specifics for the purposesof illustration, a person of ordinary skill in the art will appreciatethat many variations and alterations to the following details are withinthe scope of the invention. Accordingly, the following embodiments ofthe invention are set forth without any loss of generality to, andwithout imposing limitations upon, the claimed invention.

The present invention relates to an advanced system for observing andcharacterizing manual welding exercises and operations. This system isparticularly useful for welding instruction and welder training thatprovides an affordable tool for measuring manual welding technique andcomparing that technique with established procedures. The trainingapplications of this invention include: (i) screening applicant skilllevels; (ii) assessing trainee progress over time; (iii) providingreal-time coaching to reduce training time and costs; and (iv)periodically re-testing welder skill levels with quantifiable results.Processing monitoring and quality control applications include: (i)identification of deviations from preferred conditions in real time;(ii) documenting and tracking compliance with procedures over time;(iii) capturing in-process data for statistical process control purposes(e.g., heat input measurements); and (iv) identifying welders needingadditional training The system of the present invention provides theunique benefit of enabling the determination of compliance with variousaccepted welding procedures.

The present invention, in various exemplary embodiments, measures torchmotion and gathers process data during welding exercises using a singleor multiple camera tracking system based on point cloud image analysis.This invention is applicable to a wide range of processes including, butnot necessarily limited to, GMAW, FCAW, SMAW, GTAW, and cutting. Theinvention is expandable to a range of workpiece configurations,including large sizes, various joint types, pipe, plate, and complexshapes and assemblies. Measured parameters may include, but are notlimited to, work angle, travel angle, tool standoff, travel speed, beadplacement, weave, voltage, current, wire feed speed, arc length, heatinput, gas flow (metered), contact tip to work distance (CTWD), anddeposition rate (e.g., lbs./hr., in./run). The training component of thepresent invention may be pre-populated with specific welding proceduresor it may be customized by an instructor. Data is automatically savedand recorded, a post-weld analysis scores performance, and progress istracked over time. This system may be used throughout an entire weldingtraining program. The system may be used to provide any type of feedback(typically in real time) including, but not limited to, one or more ofin-helmet visual feedback, on-screen visual feedback, audio feedback(e.g., coaching), and welding tool (e.g., torch) visual, audio, ortactile feedback. With reference now to the Figures, one or morespecific embodiments of this invention shall be described in greaterdetail.

As shown in FIG. 1, in an exemplary embodiment of the present invention,the basic flow of information through data generating component 100,data capturing component 200, and data processing (and visualization)component 300 of weld characterization system 10 occurs in six basicsteps: (1) image capture 110; (2) image processing 112; (3) input of arcweld data 210, such as known or preferred weld parameters; (4) dataprocessing 212; (5) data storage 214; and (5) data display 310. Imagecapture step 110 includes capturing images of a target 98 (whichtypically includes at least two point markers located in a fixedgeometric relationship to one another) with one or more off-the-shelfhigh-speed-vision cameras, where the output aspect typically includescreating an image file at many (e.g., over 100) frames per second. Theinput aspect of image processing step 112 includes frame-by-frame pointcloud analysis of a rigid body that includes three or more point markers(i.e., the calibrated target). Upon recognition of a known rigid body,position and orientation are calculated relative to the camera originand the “trained” rigid body orientation. Capturing and comparing theimages from two or more cameras allows for a substantially accuratedetermination of the rigid body position and orientation inthree-dimensional space. Images are typically processed at a rate ofmore than 100 times per second. One of ordinary skill in the art willappreciate that a lesser sampling rate (e.g., 10 images/sec.) or agreater sampling rate (e.g., 1,000 images/sec.) could be used. Theoutput aspect of image processing step 112 includes creation of a dataarray that includes x-axis, y-axis, and z-axis positional data and roll,pitch, and yaw orientation data, as well as time stamps and softwareflags. The text file (including the aforementioned 6D data) may bestreamed or sent at a desired frequency. The input aspect of dataprocessing step 212 includes raw positional and orientation datatypically requested at a predetermined rate, while the output aspectincludes transforming this raw data into useful welding parameters withalgorithms specific to a selected process and joint type. The inputaspect of data storage step 214 includes storing welding trial data(e.g., as a *.dat file), while the output aspect includes saving thedata for review and tracking, saving the data for review on a monitor ata later time, and/or reviewing the progress of the student at a latertime. Student progress may include, but is not limited to, totalpractice time, total arc time, total arc starts, and individualparameter-specific performance over time. The input aspect of datadisplay step 310 includes welding trial data that may further include,but is not limited to, work angle, travel angle, tool standoff, travelspeed, bead placement, weave, voltage, current, wire-feed speed, arclength, heat input, gas flow (metered), contact tip to work distance(CTWD), and deposition rate. The output aspect involves any type offeedback including, but not limited to, one or more of visual data thatmay be viewed on a monitor, in-helmet display, heads-up display, orcombinations thereof; audio data (e.g., audio coaching); and tactilefeedback. The feedback is typically presented in real time. The trackedparameters are plotted on a time-based axis and compared to upper andlower thresholds or preferred variations, such as those trained byrecording the motions of an expert welder. One of ordinary skill in theart will appreciate that the general inventive concepts contemplateplotting the parameters on an axis that is not time-based, such asplotting or otherwise processing the parameters based on a length of theweld. Current and voltage may be measured in conjunction with travelspeed to determine heat input, and the welding process parameters may beused to estimate arc length. Position data may be transformed into weldstart position, weld stop position, weld length, weld sequence, weldingprogression, or combinations thereof and current and voltage may bemeasured in conjunction with travel speed to determine heat input.

FIGS. 2-5 provide illustrative views of weld characterization system 10in accordance with an exemplary embodiment the present invention. Asshown in FIG. 2, portable training stand 20 includes a substantiallyflat base 22 for contact with a floor or other horizontal substrate,rigid vertical support column 24, camera or imaging device support 26,and rack and pinion assembly 31 for adjusting the height of imagingdevice support 26. In most embodiments, weld characterization system 10is intended to be portable or at least moveable from one location toanother, therefore the overall footprint of base 22 is relatively smallto permit maximum flexibility with regard to installation and use. Asshown in FIG. 2-6, weld characterization system 10 may be used fortraining exercises that include any suitable arrangement of workpiecesincluding, but not limited to, flat, horizontally, vertically, overhead,and out-of-position oriented workpieces. In the exemplary embodimentsshown in the Figures, training stand 20 is depicted as a unitary orintegrated structure that is capable of supporting the other componentsof system. In other embodiments, stand 20 is absent and the variouscomponents of the system are supported by whatever suitable structuralor supportive means may be available. Thus, within the context of thisinvention, “stand” 20 is defined as any single structure or,alternately, multiple structures that are capable of supporting thecomponents of weld characterization system 10.

With reference to FIGS. 2-3, certain welding exercises will utilize aflat assembly 30, which is slidably attached to vertical support column24 by collar 34, which slides upward or downward on support column 24.Collar 34 is further supported on column 24 by rack and pinion 31, whichincludes shaft 32 for moving rack and pinion assembly 31 upward ordownward on support column 24. Flat assembly 30 includes trainingplatform 38, which is supported by one or more brackets (not visible).In some embodiments, a shield 42 is attached to training platform 38 forprotecting the surface of support column 24 from heat damage. Trainingplatform 38 further includes at least one clamp 44 for securing weldposition-specific fixture/jig 46 to the surface of the trainingplatform. The structural configuration or general characteristics ofweld position-specific jig 46 are variable based on the type of weldprocess that is the subject of a particular welding exercise, and inFIGS. 2-3, fixture 46 is configured for a fillet weld exercise. In theexemplary embodiment shown in FIGS. 2-3, first 48 and second 50structural components of weld position-specific fixture 46 are set atright angles to one another. Position-specific fixture 46 may includeone or more pegs 47 for facilitating proper placement of a weld coupon(workpiece) 54 on the fixture. The characteristics of any weld coupon 54used with system 10 are variable based on the type of manual weldingprocess that is the subject of a particular training exercise and in theexemplary embodiment shown in the FIGS. 7-8, first 56 and second 58portions of weld coupon 54 are also set at right angles to one another.With reference to FIGS. 4-5, certain other welding exercises willutilize a horizontal assembly 30 (see FIG. 4) or a vertical assembly 30(see FIG. 5). In FIG. 4, horizontal assembly 30 supports butt fixture46, which holds workpiece 54 in the proper position for a butt weldexercise. In FIG. 5, vertical assembly 30 supports vertical fixture 46,which holds workpiece 54 in the proper position for a lap weld exercise.

Data processing component 300 of the present invention typicallyincludes at least one computer for receiving and analyzing informationcaptured by the data capturing component 200, which itself includes atleast one digital camera contained in a protective housing. Duringoperation of weld characterization system 10, this computer is typicallyrunning software that includes a training regimen module, an imageprocessing and rigid body analysis module, and a data processing module.The training regimen module includes a variety of weld types and aseries of acceptable welding process parameters associated with creatingeach weld type. Any number of known or AWS weld joint types and theacceptable parameters associated with these weld joint types may beincluded in the training regimen module, which is accessed andconfigured by a course instructor prior to the beginning of a trainingexercise. The weld process and/or type selected by the instructordetermine which weld process-specific fixture, calibration device, andweld coupon are used for any given training exercise. The objectrecognition module is operative to train the system to recognize a knownrigid body target 98 (which includes two or more point markers) and thento use target 98 to calculate positional and orientation data forwelding gun 90 as an actual manual weld is completed by a trainee. Thedata processing module compares the information in the training regimenmodule to the information processed by the object recognition module andoutputs the comparative data to a display device such as a monitor orheads-up display. The monitor allows the trainee to visualize theprocessed data in real time and the visualized data is operative toprovide the trainee with useful feedback regarding the characteristicsand quality of the weld. The visual interface of weld characterizationsystem 10 may include a variety of features related to the input ofinformation, login, setup, calibration, practice, analysis, and progresstracking The analysis screen typically displays the welding parametersfound in the training regimen module, including (but not limited to)work angle, travel angle, tool standoff, travel speed, bead placement,weave, voltage, current, wire-feed speed, arc length, heat input, gasflow (metered), contact tip to work distance (CTWD), and deposition rate(e.g., lbs./hr., in./run). Multiple display variations are possible withthe present invention.

In most, if not all instances, weld characterization system 10 will besubjected to a series of calibration steps/processes prior to use. Someof the aspects of the system calibration will typically be performed bythe manufacturer of system 10 prior to delivery to a customer and otheraspects of the system calibration will typically be performed by theuser of weld characterization system 10 prior to any welding trainingexercises. System calibration typically involves two related andintegral calibration processes: (i) determining the three-dimensionalposition and orientation of the operation path to be created on aworkpiece for each joint/position combination to be used in variouswelding training exercises; and (ii) determining the three-dimensionalposition and orientation of the welding tool by calculating therelationship between a plurality of passive (e.g., reflective) or active(e.g., light emitting) point markers located on target 98 and at leasttwo key points represented by point markers located on the welding tool90.

The first calibration aspect of this invention typically involves thecalibration of the welding operation with respect to the globalcoordinate system, i.e., relative to the other structural components ofweld characterization system 10 and the three-dimensional space occupiedthereby. Prior to tracking/characterizing a manual welding exercise, theglobal coordinates of each desired operation path (i.e., vector) on anygiven workpiece will be determined. In some embodiments, this is afactory-executed calibration process that will include correspondingconfiguration files stored on data processing component 200. In otherembodiments, the calibration process could be executed in the field(i.e., on site). To obtain the desired vectors, a calibration devicecontaining active or passive markers may be inserted on at least twolocating markers in each of the various platform positions (e.g., flat,horizontal, and vertical). FIGS. 6-8 illustrate this calibration step inone possible platform position. Joint-specific fixture 46 includes firstand second structural components 48 (horizontal) and 50 (vertical),respectively. Weld coupon or workpiece 54 includes first and secondportions 56 (horizontal) and 58 (vertical), respectively. One ofordinary skill in the art will appreciate that the general inventiveconcepts extend to any suitable jig/coupon configurations andorientations. For example, in some exemplary embodiments, jig or jointcalibration could be performed using a handheld or removable device thatwould “teach” the software points (i.e., positions) to determine theoperation path. In this manner, use of the two or more points wouldallow the weldment to be oriented in any position.

Workpiece operation path extends from point X to point Y and is shown asdouble line 59 in FIG. 7. Locating point markers 530 and 532 are placedas shown in FIGS. 6 (and FIG. 8), and the location of each marker isobtained using data capturing component 100. Any suitable data capturingsystem can be used, for example, the Optitrack Tracking Tools (providedby NaturalPoint, Inc. of Corvallis, Oregon) or a similar commerciallyavailable or proprietary hardware/software system that providesthree-dimensional marker and six degrees of freedom object motiontracking in real time. Such technologies typically utilize reflectiveand/or light emitting point markers arranged in predetermined patternsto create point clouds that are interpreted by system imaging hardwareand system software as “rigid bodies,” although other suitablemethodologies are compatible with this invention.

In the calibration process represented by the flowchart of FIG. 9, table38 is fixed into position i (0,1,2) at step 280; a calibration device isplaced on locating pins at step 282; all marker positions are capturedat step 284; coordinates for the locator positions are calculated atstep 286; coordinates for the fillet operation path are calculated atstep 288 and stored at 290; coordinates for the lap operation path arecalculated at step 292 and stored at 294; and coordinates for the grooveoperation path are calculated at step 296 and stored at 298. Allcoordinates are calculated relative to the three dimensional spaceviewable by data capturing component 200.

In one embodiment of this invention, the position and orientation of theworkpiece is calibrated through the application of two or more passiveor active point markers to a calibration device that is placed at aknown translational and rotational offset to a fixture that holds theworkpiece at a known translational and rotational offset. In anotherembodiment of this invention, the position and orientation of theworkpiece is calibrated through the application of two or more passiveor active point markers to a fixture that holds the workpiece at a knowntranslational and rotational offset. In still other embodiments, theworkpiece is non-linear, and the position and orientation thereof may bemapped using a calibration tool with two or more passive or active pointmarkers and stored for later use. The position and orientation of theworkpiece operation path may undergo a pre-determined translational androtational offset from its original calibration plane based on thesequence steps in the overall work operation.

In some exemplary embodiments, data on weldments in electronic format(e.g., rendered in CAD) are extracted and used in determining positionand/or orientation of the workpiece. Additionally, an associated weldingprocedure specification (WPS) that specifies welding parameters for theweldment is also processed. In this manner, the system can map the CADdrawing and WPS for use in assessing (in real time) compliance with theWPS.

Important tool manipulation parameters such as position, orientation,velocity, acceleration, and spatial relationship to the workpieceoperation path may be determined from the analysis of consecutive toolpositions and orientations over time and the various workpiece operationpaths described above. Tool manipulation parameters may be compared withpre-determined preferred values to determine deviations from known andpreferred procedures. Tool manipulation parameters may also be combinedwith other manufacturing process parameters to determine deviations frompreferred procedures, and these deviations may be used for assessingskill level, providing feedback for training, assessing progress towarda skill goal, or for quality control purposes. Recorded motionparameters relative to the workpiece operation path may be aggregatedfrom multiple operations for statistical process control purposes.Deviations from preferred procedures may be aggregated from multipleoperations for statistical process control purposes. Important toolmanipulation parameters and tool positions and orientations with respectto the workpiece operation path may also be recorded for establishing asignature of an experienced operator's motions to be used as a baselinefor assessing compliance with preferred procedures.

The second calibration aspect typically involves the calibration of thewelding tool 90 with respect to the target 98. The welding tool 90 istypically a welding torch or gun or SMAW electrode holder, but may alsobe any number of other implements including a soldering iron, cuttingtorch, forming tool, material removal tool, paint gun, or wrench. Withreference to FIGS. 10-11, welding gun/tool 90 includes tool point 91,nozzle 92, body 94, trigger 96, and target 98. A tool calibration device93, which includes two integrated active or passive point markers in theA and B positions (see FIG. 11) is attached to, inserted into, orotherwise interfaced with the nozzle 92. For example, the tool point 91can be machined off so that the tool calibration device 93 can bethreaded into the nozzle 92 of the welding tool 90 for calibrationpurposes.

In another exemplary embodiment, the tool calibration device 93 isaffixed to a sleeve 1100 as shown in FIG. 11A. The sleeve 1100 can beshaped and sized so as to fit over at least a portion of the nozzle 92of the welding tool 90. In some embodiments, the sleeve 1100 can fitover the nozzle 92 without requiring removal of the tool point 91. Insome exemplary embodiments, the sleeve 1100 fits over or otherwiseinterfaces with a part of the welding tool 90 other than the nozzle 92.The sleeve 1100 removably connects to the welding tool 90 in anysuitable matter, for example, by friction fit, threads, etc. The sleeve1100 can be shaped and sized so as to fit over a plurality of differentwelding tools, without requiring modification of said welding tools. Inthis manner, the sleeve 1100 and the attached tool calibration device 93form a type of “universal” tool calibration device.

Additionally, a rigid body point cloud (i.e., a “rigid body”) isconstructed by attaching active or passive point markers 502, 504, and506 (and possibly additional point markers) to the upper surface oftarget 98.

As described herein, other point marker placements are possible and fallwithin the scope of the general inventive concepts. Target 98 mayinclude a power input if the point markers used are active and require apower source. Data capturing component 200 uses a tracking system (e.g.,the aforementioned Optitrack Tracking Tools) or similarhardware/software to locate the rigid body and point markers 522 (A) and520 (B), which represent the location of a tool vector. These positionscan be extracted from the software of system 10 and the relationshipbetween point markers A and B and the rigid body can be calculated.

In the exemplary calibration process represented by the flowchart ofFIG. 12, weld nozzle 92 and the contact tube 91 are removed at step 250;the calibration device is inserted into body 94 at step 252; weld tool90 is placed in the working envelope and rigid body (designated as “S”in FIG. 12) and point markers A and B are captured by data capturingcomponent 100; and the relationships between A and S and B and S arecalculated at step 256; with relationship data for A_(s) being stored at258 and relationship data for B_(s) being stored at 260.

In one embodiment of this invention, calibration of the tool point andtool vector is performed through the application of two or more passiveor active point markers to the calibration device at locations along thetool vector with a known offset to the tool point. In anotherembodiment, calibration of the tool point and tool vector is performedby inserting the tool into a calibration block of known position andorientation relative to the workpiece. For example, calibration of thetool point and tool vector can be performed by inserting the tool point91 into the weld joint in a particular manner.

With regard to the rigid body defined by the point markers (e.g., 502,504, 506), in one embodiment, the passive or active point markers areaffixed to the tool in a multi-faceted manner so that a wide range ofrotation and orientation changes can be accommodated within the field ofview of the imaging system. In another embodiment, the passive or activepoint markers are affixed to the tool in a spherical manner so that awide range of rotation and orientation changes can be accommodatedwithin the field of view of the imaging system. In still anotherembodiment, the passive or active point markers are affixed to the toolin a ring shape so that a wide range of rotation and orientation changescan be accommodated within the field of view of the imaging system.

Numerous additional useful features may be incorporated into the presentinvention. For example, for purposes of image filtering, band-pass orhigh-pass filters may be incorporated into the optical sequence for eachof the plurality of digital cameras in data capturing component 200 forpermitting light from only the wavelengths which are reflected oremitted from the point markers to improve image signal-to-noise ratio.Spurious data may be rejected by analyzing only image informationobtained from within a dynamic region of interest having a limitedoffset from a previously known rigid-body locality. This dynamic regionof interest may be incorporated into or otherwise predefined (i.e.,preprogrammed as a box or region of width x and height y and centered onknown positions of target 98) within the field of view of each digitalcamera such that image information is only processed from thispredefined region. The region of interest will change as the rigid bodymoves and is therefore based on previously known locations of the rigidbody. This approach allows the imaging system to view only pixels withinthe dynamic region of interest when searching for point markers whiledisregarding or blocking pixels in the larger image frame that are notincluded in the dynamic region of interest. Decreased processing time isa benefit of this aspect of the invention.

In some embodiments of the present invention, the position andorientation of the operation path, or a predetermined segment thereof,relative to the three-dimensional space viewable by the imaging systemis obtained from a three-dimensional CAD model, the coordinate system ofwhich is known relative to the coordinate system of the imaging system.The three-dimensional CAD model may also contain a definition of linearor curvilinear points which define the operation path segment and atleast three calibration points are located on both the three-dimensionalCAD model and on the fixture. A position and orientation shift may beapplied to the three-dimensional CAD model by measuring the position ofthe at least three calibration points on the fixture with the imagingsystem and then comparing the measurements to the original calibrationpoints of the three-dimensional CAD model. In other embodiments, theposition and orientation of the linear or curvilinear operation path, ora predetermined segment thereof, relative to the three-dimensional spaceviewable by the imaging system may obtained using a three-dimensionalCAD model, wherein the coordinate system of the three-dimensional CADmodel relative to the coordinate system of the imaging system ispredetermined, and wherein the weld locations on the three-dimensionalCAD model are pre-defined. Regarding the CAD model creation, there istypically a one-to-one relationship between the CAD model and the partin question and a sequence of calibration may be an aspect of thewelding exercise. The model exists is virtual space and the userinstructs the system as to the location of the two points. A linkage iscreated to eliminate any variance between the CAD model and the part orparticular datum on tooling is utilized. A procedure to teach the systemoffline may also be included.

One definition of an operation path for this invention describes asingle continuous path for operation. In certain embodiments, theoperation path is divided into separate segments for welds that traversecorners or change in general direction. In this context, points make upan operation path segment (at least two), and contiguous operation pathsegments make up an operation path chain. Thus, the position andorientation of the operation path may be made up of one or moreoperation path segments that form a chain, and consecutive segmentsshare an operation path point at the end of one segment and the start ofthe next segment. In such embodiments, the system provides the abilityto move between multiple calibration planes; each operation planedepends on which calibration plane is being utilized, and each operationpath is tied to a predetermined coordinate system.

Furthermore, the exemplary weld characterization system 10 of FIG. 2 canbe modified to better process the case of a round (e.g., circular)operation path, such as with pipe welding, and/or a more complexassembly of workpieces.

As shown in FIG. 13, the weld characterization system 10 includes atraining stand similar or identical to the stand 20 shown in FIG. 2,having the substantially flat base 22. Furthermore, a frame 1302, cage,or the like is also interfaced with the stand 20 (either directly orindirectly). For example, the frame 1302 could be connected to thecamera or imaging device support 26.

In some embodiments, the frame 1302 can be readily separated from thestand 20 to promote the portability of the system 10. The frame 1302includes one or more legs 1304 for further supporting the frame 1302,along with the stand 20. The legs 1304 may be height-adjustable. Theframe defines various locations as which cameras (e.g., digitalhigh-speed-vision cameras) can be mounted. As shown in FIG. 13, aplurality of cameras 1306 are mounted on the frame 1302 as variouslocations (and elevations) around the workpiece 54. In this manner, theframe 1302 at least partially surrounds the workpiece 54. In someexemplary embodiments, at least half of the workpiece 54 (i.e., 180degrees in the case of a pipe) is surrounded by the frame 1302. In someexemplary embodiments, at least 75% of the workpiece 54 (i.e., 270degrees in the case of a pipe) is surrounded by the frame 1302. In someexemplary embodiments, substantially all (i.e., ˜100%) of the workpiece54 is surrounded by the frame 1302.

The cameras 1306 form part of the data capturing component 200. In someexemplary embodiments, the weld characterization system 10 includes 2 ormore (e.g., 2-20) cameras. In some exemplary embodiments, the weldcharacterization system 10 includes 4 or more cameras, 5 or morecameras, 6 or more cameras, 7 or more cameras, 8 or more cameras, 9 ormore cameras, 10 or more cameras, 11 or more cameras, or 12 or morecameras. In some exemplary embodiments, the weld characterization system10 includes at least 4 cameras, at least 5 cameras, at least 6 cameras,at least 7 cameras, at least 8 cameras, at least 9 cameras, at least 10cameras, at least 11 cameras, or at least 12 cameras. After calibrationof the cameras (as needed), the weld coupon 54 and the welding tool 10are calibrated as described herein. The distribution of the cameras 1306around the workpiece 54 (e.g., a pipe) allow for the accurate trackingof welding operations on the workpiece 54.

While the present invention has been illustrated by the description ofexemplary embodiments thereof, and while the embodiments have beendescribed in certain detail, it is not the intention of the Applicant torestrict or in any way limit the scope of the appended claims to suchdetail. Additional advantages and modifications will readily appear tothose skilled in the art. Therefore, the invention in its broaderaspects is not limited to any of the specific details, representativedevices and methods, and/or illustrative examples shown and described.Accordingly, departures may be made from such details without departingfrom the spirit or scope of the applicant's general inventive concept.

1. A system for characterizing welding operations, comprising: (a) adata generating component, wherein the data generating component furtherincludes: (i) a fixture, wherein the geometric characteristics of thefixture are predetermined; (ii) a workpiece adapted to be mounted on thefixture, wherein the workpiece includes at least one joint to be welded,wherein the vector extending along the joint to be welded defines anoperation path, and wherein the operation path is linear, curvilinear,circular, or a combination thereof; (iii) at least one calibrationdevice, wherein each calibration device further includes at least twopoint markers integral therewith, and wherein the geometric relationshipbetween the point markers and the operation path is predetermined; and(iv) a welding tool, wherein the welding tool is operative to form aweld at the joint to be welded, wherein the welding tool defines a toolpoint and a tool vector, and wherein the welding tool further includes atarget attached to the welding tool, wherein the target further includesa plurality of point markers mounted thereon in a predetermined pattern,and wherein the predetermined pattern of point markers is operative todefine a rigid body; and (b) a data capturing component, wherein thedata capturing component further includes an imaging system forcapturing images of the point markers; and (c) a data processingcomponent, wherein the data processing component is operative to receiveinformation from the data capturing component and then calculate: (i)the position and orientation of the operation path relative to thethree-dimensional space viewable by the imaging system; (ii) theposition of the tool point and orientation of the tool vector relativeto the rigid body; and (iii) the position of the tool point andorientation of the tool vector relative to the operation path.
 2. Thesystem of claim 1, wherein the imaging system further includes aplurality of digital cameras.
 3. The system of claim 2, wherein at leastone filter is incorporated into the optical sequence for each of theplurality of digital cameras for permitting light from only thewavelengths which are reflected or emitted from the point markers forimproving image signal-to-noise ratio.
 4. The system of claim 2, whereinthe imaging system further includes at least one dynamic region ofinterest viewable by the plurality of digital cameras, wherein thedynamic region of interest is determined by use of previously knownpositions for the rigid body, and wherein image information is gatheredand processed only from within the dynamic region of interest.
 5. Thesystem of claim 2, wherein the plurality of digital cameras includes atleast one digital camera positioned above the workpiece and at least onedigital camera positioned below the workpiece.
 6. The system of claim 1,wherein the workpiece comprises a pipe.
 7. The system of claim 1,wherein the position and orientation of the operation path is calibratedusing at least two point markers integral to a calibration device whichis placed at a known translational and rotational offset to the fixture,and wherein the fixture holds the workpiece at a known translational androtational offset to the operation path.
 8. The system of claim 1,wherein the position and orientation of the operation path is calibratedusing at least two point markers located on a fixture that holds theworkpiece at a known translational and rotational offset to theoperation path.
 9. The system of claim 1, wherein the operation path isnon-linear, wherein the position and orientation of the operation pathin three-dimensional space may be mapped using a calibration device thatincludes at least two point markers, and wherein the operation pathdictates the placement of the calibration device at multiple, differentpoints thereon.
 10. The system of claim 9, wherein the operation path iscircular.
 11. The system of claim 1, wherein the position andorientation of the operation path undergoes a predeterminedtranslational and rotational offset from its original calibration planebased on predetermined sequence steps included in the overall systemoperation.
 12. The system of claim 1, wherein the position andorientation of the operation path, or a predetermined segment thereof,relative to the three-dimensional space viewable by the imaging systemis obtained from a three-dimensional CAD model, the coordinate system ofwhich is known relative to the coordinate system of the imaging system.13. The system of claim 12, wherein the three-dimensional CAD modelcontains a definition of linear or curvilinear points which define theoperation path segment.
 14. The system of claim 12, wherein at leastthree calibration points are located on both the three-dimensional CADmodel and on the fixture.
 15. The system of claim 14, wherein a positionand orientation shift is applied to the three-dimensional CAD model bymeasuring the position of the at least three calibration points on thefixture with the imaging system and then comparing the measurements tothe original calibration points of the three-dimensional CAD model. 16.The system of claim 1, wherein the position and orientation of theoperation path is made up of one or more operation path segments thatform a chain, and wherein consecutive segments share an operation pathpoint at the end of one segment and the start of the next segment. 17.The system of claim 1, wherein calibration of the tool point and toolvector of the welding tool is performed using two or more point markersintegrated into a removable calibration device, and wherein the pointmarkers in the calibration device are located along a tool vector thathas a known offset to the tool point of the welding tool.
 18. The systemof claim 1, wherein calibration of the tool point of the welding tool isperformed by inserting the tip of the welding tool into a calibrationdevice, the position and orientation of which relative to the workpieceis predetermined.
 19. The system of claim 18, wherein the systemcalculates values for at least one of tool position, orientation,velocity, and acceleration with respect to the operation path, whereinthese values are then compared to predetermined preferred values todetermine deviations from known and preferred procedures, and whereinsuch deviations are used for at least one of assessing skill level,providing feedback for training, assessing progress toward a skill goal,and quality control purposes.
 20. The system of claim 1, wherein thepoint markers defining the rigid body are affixed to the welding tool ina multi-faceted configuration that accommodates a wide range of rotationand orientation changes of the welding tool when in use.
 21. The systemof claim 1, wherein the point markers defining the rigid body areaffixed to the welding tool in a spherical configuration thataccommodates a wide range of rotation and orientation changes of thewelding tool when in use.
 22. The system of claim 1, wherein the passiveor active point markers are affixed to the welding tool in a ringconfiguration that accommodates a wide range of rotation and orientationchanges of the welding tool when in use.
 23. The system of claim 1,wherein the system calculates values for at least one of tool position,orientation, velocity, and acceleration with respect to the operationpath, wherein these values are then compared to predetermined preferredvalues to determine deviations from known and preferred procedures, andwherein such deviations are used for at least one of assessing skilllevel, providing feedback for training, assessing progress toward askill goal, and quality control purposes.
 24. The system of claim 1,wherein the fixture includes a frame; wherein the frame at leastpartially surrounds the workpiece; wherein the frame includes aplurality of digital cameras mounted thereon; and wherein each digitalcamera captures images of the workpiece from a different angle.